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ABSTRACT 

This study attempted to identify relationships 
between three measures of supervision effectiveness (satisfaction, 
supervisor competence, and supervision's contribution to skill 
development) and 48 supervisor behaviors as presented in the 
Supervision Questionnaire-Revised (SQ--R) and to compare the results 
with those of a previous study conducted on the Supervision 
Questionnaire (SQ) . Subjects were 42 masters and doctoral clinical 
psychology students at a small southeastern university. Each received 
the SQ-R in his/her mailbox following completion of a 150--hour 
therapy practicum. Correlational analysis revealed 23 of the 48 
supervisor behaviors correlated significantly (p<.Ol) with measures 
of satisfaction, supervisor competence, and contribution to skills. 
Of these 23 behaviors, 15 matched those identified in the earlier 
study using the SQ. Suggested by the results is that approximately 
50% of the supervisor behaviors identified on the SQ-R are 
significantly related to supervision effectiveness &s perceived by 
supervisees. Greater than 50*i of these behaviors were consistent with 
those earlier identified using the SQ, suggesting a moderate degree 
of continuity between instruments. This continuity increases 
confidence in previously published results, and suggests that the 
revised form is measuring variables similar to the those measured by 
the original form. (Author) 
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.\BSTRACT 

This study attempted to identily relationships between three measures of 
supervision effectiveness (satisfaction, supervisor competence, and supervision's 
contribution to skill development) and 48 supervisor behaviors as presented in the 
Supervision Questionnaire-Revised (SQ-R; and, to compare these results with those of a 

previous study (Worthington and Roehlke, 1979) conducted on the Supervkuin 

Questionnaire (SQ). Subjects were forty-two (25 female and 17 male) masters and 
doctoral clinical psychology students at a small southeastern university. Each received the 
SQ-R in his/her mailbox foUowing the completion of a 150 hour therapy practicum. 
Correlational analysis revealed 23 of the 48 supervisor behaviors correlate significantlv 
(p*-.Oi) with measures of satisfaction, supervisor competence, and contributions to skills. 
Of these 23, 15 matched those identified in the earlier study using the SQ (Worthington 
and Roehlke, 1979). Suggested by the resuUs is that approximately 50% of the supervisor 
behaviors identilied on the SQ-R are significantly related to supeivision efl'ectiveness as 
perceived by supervisees. Greater than 50% of these behaviors were consistent with those 
identified earlier using the SQ, suggesting a moderate degree of continuity' between 
instruments. This continuity increases confidence in previously published results, and 
suggests iliat the revised ionn is measuring variables similar to the original form. 
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Factors That Measure Supervision Effectiveness: A Comparison 

of the SQ and SQ-R. 

A major goal of training programs leading to a doctoral 
degree in professional psychology is the development of 
psychotherapy skills. These skills are presumably acquired 
through a series of formal learning contracts (courses, 
workshops, seminars) and supervised experiences (practica 
and internships). The formal learning is not covered in 
this review. This research focuses on the complex problem 
of supervision for students. 

In general, there are three main variables to consider 
when studying the supervision process: The supervisor, the 
supervisee, and the supervision environment. The supervisor 
has been studied in terms of perceived personal 
characteristics and how these impact on supervision. Such 
characteristics include dimensions of the Interpersonal 
Influence Model (Expertness, Trustworthiness, and 
Attractiveness) as examined by Heppner and Handley (1982) 
and Cross and Brown (1983), gender of the supervisor 
(Worthington and Stern, 1985; Horowitz, 1990), supervisor 
behaviors within supervision (Worthington and Roehlke, 1979; 
Heppner and Roehlke, 1984), and the supervisor's experience 
level (Goodyear and Robyak, 1982; Worthington, 1984; and 
Zucker and Worthington, 1986). 

Studies focusing on the second variable, the supervisee 
have emphasized the supervisee's perception of the degree to 
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which their supervisor possesses expertness, 
trustworthiness, and attractiveness (Heppner and Roehike, 
1984), and the needs supervisees have in the supervision 
process (Reising and Daniels, 1983; Stoltenberg, Pierce, and 
McNeill, 1987). 

The thi rd variable, supervis ion envi ronment, attempts 
to utilize what is known about supervisor characteristics 
and supervisee needs to create circumstances that facilitate 
the supervision process. Studies have shown that 
supervisees will expect/need different skills training and 
will have different expectations of the supervisory 
relationship depending on the number of supervised practica 
a supervisee had had. For example, Heppner and Roehike 
(1984) showed that while a supportive supervisory 
relationship was desired at all levels, more advanced 
students (third and fourth practica) were able to process 
confrontation and negative feedback more readily. 

In summary, the three essential factors in studying the 
supervision process include the supervisor, the supervisee, 
and the supervision environment. The influence of personal 
characteristics of the supervisor and supervisee on the 
supervision process have been studied both separately and as 
interacting variables. It is this interaction between what 
the supervisor and supervisee brings to the supervision that 
creates the third variable, supervision environment. Thus, 
whether or not the supervision environment will facilitate 
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the supervision process may depend on the match between 
supervisor and supervisee characteristics. 

The question becomes how to study each of the 
variables. Traditionally, supervision was conceptualized in 
terms of the prominent psychological theories. Thus, the 
first attempt at supervision involved the student supervisee 
in psychoanalysis as a means of acquiring therapy skills and 
knowledge (Worthington, 1987). Others would focus on the 
relationship qualities, or experiencing the supervision 
process in the here and now, as a means for developing the 
supervisee's skills and knowledge (Bernard and Goodyear, 
1992). 

More recently, two other approaches, less related to a 
specific theory in psychology, have been employed when 
studying the supervi sion- process . The first of these two is 
a developmental approach and includes numerous models of 
supervision (Hogan, 1964; Littrell, Lee-Borden, and Lorenz, 
1979; Stoltenberg, 1981; and Loganbill, Hardy, and Delworth, 
1982). In essence, these models delineate the developmental 
progression of different components of the supervision 
process (supervisee , supervi so r , supervi sion envi ronment ) 
separately, as well as in combination. The assumption is 
that there is a fairly consistent pattern of supervisee 
development as experience is gained. The same is presumed 
about the supervisor as well as appropriate learning 
envi ronments . 
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The second approach might be classified as a 
statistical approach. Basically, this method involves 
analyzing ratings of a number of specific behaviors to 
determine: 1. More general factors that contribute to 
facilitating the supervision process; and, 2. How the more 
general factors are related to outcome measures of 
satisfaction, contribution to skills, and supervisor 
competence. Worthington and Roehlke (1979) combined these 
three variables giving the single measure the label of 
"supervision effectiveness". What this approach lacks in 
comprehensiveness (ie. considering only supervisor behaviors 
from the whole process) it gains in utility. It provides 
statistically derived information at two levels: general 
(factors) and specific (individual behaviors). The value of 
direct, concrete information becomes even more obvious 
against a background of diverse supervision models, and the 
fact that supervision research is still in its "adolescence" 
(Bernard and Goodyear, 1992). An early study using the 
statistical model was conducted by Worthington and Roehlke 
(1979) as they examined ratings on the Supervision 
Questionnaire (SQ). Surveying both supervisors and 
supervisees they found that supervisors perceived 
supervision as primarily providing feedback tosupervi sees . 
In contrast, for beginning counselors, three factors were 
important: a ple.isant supervisory relationship; structured 
supervision, especially initially; and, specific teaching of 
skills and encouragement to try these newly learned skills. 
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The Supervision Questionnaire (SQ) has become one of 
the most frequently used questionnaires in supervision 
research (Worthington, 1987). Its utility has already been 
documented. There is much to suggest its reliability and 
validity as well. Worthington (1984) provides a comparison 
of factor analytical studies of the supervision 
questionnaire (SQ) at two different times, as well as 
between the SQ and factor analytic work of Reising and 
Daniels (1983). In general, there is emerging consensus 
about what factors influence the supervision process. For 
example, several general factors were found related to 
change with experience (independence) and fostering change 
(support and encouragement) - 

As research on supervision continues, the knowledge and 
the instruments to measure the process change. The most 
frequently used instrument, the SQ, has been revised. With 
the rewording of several items, and the addition of eight 
new items much of the SQ~Revised is similar to its 
predecessor, the SQ. However, as with any other measurement 
instrument, revisions require a re-establishment of that 
instruments vital characteristics ~ reliability and 
vandity. In an unpublished manuscrTpt Worthington (1985) 
reported the six week Test-Retest reliability of the SQ-R. 
J^i^ results indicate a statistical ly significant correlation 
for ratings of satisfaction and impact of supervision on 
counsel i ng abi 1 i ty . The thi rd measure contr i buti ng to 
supervision effectiveness, competence of supervisor, did not 
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evidence a statistically significant level of reliability. 
This study suggests the SQ-R does possess some reliability, 
as well as some differences with the SQ. 

The first purpose of the present study is to examine 
what supervisor behaviors supervisees believe to be 
significantly related to satisfaction with supervision, 
competence of supervisor, and contribution to counseling 
skills. In part, this study parallels one of the original 
works on the Supervision Questionnaire (Worthington and 
Roehlke, 1979). Differences are that this study uses the 
SQ-R rather than the SQ, and combines practica students from 
several levels as opposed to "beginning level" only. The 
second purpose of this study is to compare results from 
Worthington and Roehlke (1979) with results from the present 
study. Differences are expected given changes in the 
instrument, and population sampled. However, one would 
expect the discrepancies not to be very large as the changes 
are minimal . 

Method 

Sub^iects 

Subjects were forty two (25 female and 17 male) masters 
(15) and Psy D. (27) clinical psychology students at a small 
Southeastern Umvers"'ty. Subjects were selected following 
the completion of a supervised practicum between the fall 
semester of 1S90 and the spring semester of 1992. Fifteen 
subjects just completed their first supervised practica, 
seven their second, nine their third, and eleven their 
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fourth. Return rates were (71%) 15 out of 21, (48%) 11 out 
of 23, and (25%) 1 out of 4 for the fall of 1990 and spring 
and summer of 1991, respectively. In the fall of 1991 and 
spring of 1992, return rates were (69%) 9 of 13, and (66%) 6 
of 9, Overall, 42 out of 70 (60%) packets were returned. 
Subjects ranged in age from 25 to 60. Theoretical 
orientation was not assessed. 
Materials 

Containing 48 supervisor behavior items and three 
supervi si on ef fecti veness i tems , the Supervi si on 
Questionnaire - Revised (Worthi ngton , 1984) was designed to 
assess supervisee perceptions of supervisor behaviors 
constituting effective supervision. Completed at the end of 
the semester, the SQ-R rates effectiveness using a 7 point 
Likert scale on three dimensions: 1 "Totally Unsatisfied" to 
7 "Totally Satisfied"; 1 "Totally Incompetent" to 7 "Totally 
Competent"; and, 1 "Had almost no effect" to 7 "Had a very 
large effect". Forty-eight supervisor behaviors were rated 
on a 5 point Likert scale indicating the frequency with 
which each behavior was performed (1 "never descriptive" to 
5 "perfectly descriptive"), 
p rocedure 

Fo 1 1 owi ng each semester , the names of masters and 
doctoral students having just completed a supervised 
practica were obtained from the department secretary. Each 
student listed as having completed a practicum received a 
packet of questionnaires in her/his mailbox. Of the 
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questionnaires mentioned, only the Supervision 
Questionnaire-Revised (SQ-R) was analyzed in the present 
study. The complete packet contained the Supervision 
Questionnaire-Revised (SQ-R), Bem Sex Role Inventory, 
Supervisee Needs Questionnaire, an informed consent letter, 
results request form, and instruction sheet. The 
instruction sheet informed subjects of the nature of the 
study ("Examining supervisee's perceptions of the 
supervision process".), time required to complete the 
questionnaires, and a note that supervisors and faculty were 
not to see the returned data. Also, detailed procedural 
instructions were listed: Subjects were instructed to 
complete each numbered questionnaire contained in the 
numbered manila envelope and sign their name to the back of 
the manila envelope. Subjects were instructed that after 
completing the questionnaires, they were to place them into 
the manila envelope, seal it, and return it to the research 
assistant. As each student returned the manila envelope, 
the numbered questionnaires were dissociated from the 
envelope, identifying the returnees by number only. Follow- 
up letters were distributed after a two week period, and 
follow-up phone calls were made after a four week time 
period. This procedure was followed for all semesters. 

Resu 1 ts 

Results of the correlations between forty eight 
supervisor behaviors and three measures of supervision 
effectiveness - Satisfaction, Supervisor Competence, and 
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Supervision's Contribution to Skills -are presented in table 
1 . • * 

Table 1 

Correlations between three measures of supervision, 
effectiveness and the frequencies of 48 supervisor behaviors 
as perceived by supervisees. 



Sati sf action 
with super- 
Supervisor Behavi or vi sion 



Supervisor Supervision ' s 
Competence contri bution 
to improved 
counsel i ng 



1. Established good .5324** 
rapport • 

2. Clear, jointly .4651* 
establ i shed goal s . 

3. More instruction .2283 
in initial session 

vs. later one. 

4. Observed your 
counseling at least .0250 
once ( 1 i ve ) . 

5. Observed three -.1198 
videotapes this term. 

6. Listened to three .0482 
audiotapes. 

7. Provided with re- .2098 
1 event 1 i terature . 

8. Feedback about .5153** 
posi ti ve behavi ors . 

8b. Feedback about .1570 
non-faci 1 itative 
behav i ors . 

9. Sensitive to -.1750 
d i f f erences between 

action and words. 

10. Modeled good -.0172 
task oriented skills. 



. 5020** 



2023 



2168 



079: 



-.3309 



1888 



3534 



5111** 



0000 



182 



.3967* 



.4065* 



.2618 



0478 



-.2351 



.3097 



2637 



3181 



.3237 



4295* 



,0251 
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Satisfaction 

11. Gave appropriate .5212** 
direct suggestions. 

12. Allowed co-counse- .0949 
1 ing, view audio- 
videotapes. 

13. Available at times .0558 
other than scheduled. 

14. Used relationship -.0098 
to demonstrate pr i nci - 

pies of counseling. 

15. Joint concept- .2467 
ualization of clients. 

16. Encouraged .4815* 
experimentation . 

17. Suggested ways to .0428 
get conceptual i zation 
accepted. 

18. Used humor. .3163 

19. Used labels: 

Ef f ecti ve , i nef f ec- 

tive vs right or 

wrong. .2825 

20. Helped to develop .5105** 
sel f~conf i dence . 

21 . Helped real i ze .4563* 
that trying new 

skills is awkward . 



22. Confronted when 
appropr i ate . 

23. Helped assess 
self on strengths. 

23b. Helped assess 
sel f on weaknesses 

24. Evaluated at mid 
semester . 



3660 



5984** 



50 1 6** 



1821 



Competence 
.4745 



,0338 



Improvement 
.4793* 



1530 



1 601 



1249 



.3806 



.3501 



.0789 



.3835* 



. 1901 
.3968* 

.3434 

.3162 
. 4399^ 

. 1420 



2962 



4243* 



. 4770* 



4344** 



.2862 



2650 



-. 1854 
.3465 

.4535* 

.2997 
. /948 
. 388S* 



1444 
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25. Re-negotiated 
goals at mid 
semester . 

26. Used name once 
per session. 

27. Provide sugges- 
tions for alterna- 
tive conceptual i zat i on . 



1074 



Sati sf acti on 
-.0920 



0804 



0000 



Competence 
.0000 



0756 



1 146 



Improvement 
-.0614 



2881 



28 . Provi de sugges- 
t i ons for al ter native 
i nterveni ng , 



1035 



- .0830 



5872* 



29 . Discusses exper- 
ience in practica 
class. 



-.0687 



-.0714 



1 1 78 



30. Gave appropriate .4996** 
emot i onal support . 

< 31. Taught counseling .2460 
behaviors , f aci 1 i tate 
style. 

32. Encouraged own .6049** 
style . 

33. Helped with .2159 
personal probl ems . 

34. Role played in- -.0293 
tervent i on techni ques . 



.4576* 



.51 17** 



.6159** 



.0719 



.3154 



2419 



. 6710** 



5201 



,441 5** 



,4806* 



35. Helped deal with 
def ensi veness . 



.0882 



. 1429 



.5429** 



36. Share own client 
experiences. 



2171 



.2852 



,2468 



37. Consulted you when 
there was an emergency 
wi th your cl ient . 



1667 



..2043 



3002 



38. Missed no more 
than one session. 



-.0384 



39. Session lasted 
at least 50 minutes. 



-.0500 



0936 



1521 



40. Porty five minutes 
spent with counselor/ 
cl ient. 



2290 



0637 



0244 
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Satisfaction 

41 . Session focused 

on supervisory .2512 
relationshi p . 

42. Session focused on 
counsel i ng content , .0705 

43. Session focused on 
dynamics of clients .0202 
personal i ty . 

44. Made easy to give .4524* 
process feedback. 

45. Helped develop .0417 
intake interview 

ski lis. 

46. Helped with 
consultation, case .0683 
di sposi tion . 



n = 42 
* = p < .01 
** = p < .001 

Twelve supervisor behaviors were statistically significantly 
correlated with supervision satisfaction at a level of .01 
or greater. Eight of the 12 were correlated at the .001 
level. A total of ten supervisor behaviors were 
statistically si gn i f i cant 1 > correlated with perceived 
competence of the supervisor; five at the .01 level and fTve 
at the .001 level. Regarding perceptions of the degree to 
which supervision lead to improved skills, fourteen 
supervisor behaviors were statistically significantly 
correlated at the .01 level or greater. Only two reached 
significance at the .001 level. 



Competence 
.0000 

.0302 
-.0887 

. 5266** 
-.0521 

.0789 



Improvement 
. 1640 

.0380 
.2486 
.2378 
.2073 

. 2329 
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Comparisons between a previous study using the SQ, and 
the present SO-R study reveal a moderate level of 
similarities. Overall, in the previous SQ study, thirty six 
supervisor behaviors were statistically significantly 
correlated with the three measures of supervision 
effectiveness. Using the SQ~R, the present study reveals 
that fifteen of the thirty six {42%) supervisor behaviors 
overlapped with the prior study. Specifically, of the 
fourteen SQ behaviors that reached significance for the 
variable of satisfaction, six also reached significance as 
presented on the SQ-R. For the variable of supervisor 
competence, there were only three SQ-R items that ov>;rlapped 
the eleven SQ items that correlated significantly. The 
highest percentage of overlap between SQ and SQ-R items was 
on the variable of improved counseling. Of eleven items 
significantly correlated in the SQ study, this SQ-R study 
found six of the eleven (54%) to still correlate 
si gni f icantl y . 

Satisfaction with Supervision: 

Supervi see rated superv i sor behav i ors wh i ch cor re 1 a ted 
significantly with satisfaction with supervision (p < .01) 
in both the SO and present, SQ-R study are the fol lowing: 1, 
Helping superv i sees deve 1 op the i r own sty 1 es ; 2 . 
Establishing good rapport with supervisees; 3. Helping 
supervisees develop self-confidence; 4. Giving feedback 
about counseling strengths; 5. Giving feedback about 
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positive counseling behaviors; and, 6. Reassuring counselors 
that new counseling skills seem awkward initially. 

Supervisor behaviors correlating with satisfaction at a 
significance level of p < .01 in the SQ-R study, but not in 
the SO are: 1. Gave appropriate, direct suggestions; 2. 
Helped assess self on weaknesses; 3. Gave appropriate 
emotional support; 4. Clear, jointly established goals; 5. 
Encouraged experimentation; 6. Made easy to give process 
feedback. 

Supervisor Competence: 

Those supervisor behaviors that correlated 
significantly with supervisor competence in both the SO and 
SQ-R studies include the following: 1. Used humor during 
supervisory sessions; 2. Helping supervisees develop their 
own styles; 3. Helping them develop self-confidence. Seven 
other supervisor behaviors reached significance in the SQ-R 
study, but did not match with results from the SQ study. 
These include: 1. Established good rapport; 2, Provide 
feedback about positive behaviors; 3. Encouraged own style; 
4. Made easy to give process feedback; 5. Gave appropriate, 
d i rect suggest! ons ; 6 . He 1 ped assess se 1 f on strengths ; 
afid, 7. Gave appropriate emotional support. 
Improved Counsel ing: 

Supervisor behaviors which correlated significantly 
with improved counseling skills in both the SQ and SQ-R 
include: 1. Evolving case conceptualizations jointly with 
supervisees; 2. Using the relationship to demonstrate 
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principles of counseling; 3. Being sensitive to differences 
between the way supervisees talk about counseling and the 
way they behave during counseling; 4. Reassuring supervisees 
that new counseling skills often seemed awkward initially; 
5. Establishing good rapport with supervisees; and, 6. 
Helping supervisees develop their own style. 

Eight other supervisor behaviors were significantly 
correlated with improved skills on the SQ-R results but not 
those of the SQ. Behaviors were: 1. Helped deal with 
def ensi veness ; 2. Worked on clear, jointly established 
goals; 3. Gave appropriate, direct suggestions; 4. 
Encouraged experimentation; 5, Helped assess self on 
weaknesses; 6. Provided suggestions for alternative 
interventions; 7. Helped with personal problems that 
i nterf ere ; and , 8 . Role pi ayed i nterventi on techniques • 

Discussion 

The purposes of the present study were to identify 
supervisee rated supervisor behaviors that correlate with 
measures of effectiveness (satisfaction, supervisor 
competence, and supervision's effect on improved counseling 
skills); and, to compare results of a previous study using 
the Supervision Questionnaire (SQ) with the current results 
of the SQ-R. The questions to be answered were: What 
supervisor behaviors are important to effective supervision 
as perceived by supervisees? , and, To what degree is the 
revised form of the SQ similar to the original? 



ERLC 



18 



Supervi si on Ef f ecti veness 

17 

In general, the answer to the first question is that 
many supervisor behaviors as listed on the SQ-R are of 
importance. Twenty six of the forty eight behaviors were 
significantly correlated; some were correlated with all 
three measures, whereas others were not. A general trend is 
that what seems likely intuitively was supported 
empirically. That is, behaviors such as established good 
rapport, and helped to evaluate self on strengths and 
weaknesses were more strongly correlated than items such as 
used humor, sessions length, or used ones name in the 
session . 

How do these results compare with outcomes of previous 
research using the SQ? To answer this question, a 
comparison is made between the present SQ-R study, and one 
of the original studies using the SQ (Worthington and 
Rohlke, 1979). This latter study was chosen for comparison 
because it most resembles the present study, and it employs 
the Supervision Questionnaire prior to its revision. Thus, 
with some degree of confidence, it can be determined what 
similarities and differences might be expected between the 
SQ and the SQ-R given the revision of this instrument. 

In general, there was a moderate degree of overlap 
between the SQ and the SQ~R. Of the 26 s i gn i f 1 cant 1 > 
correlated items in the present SQ~R study, fifteen were the 
same as those in the original, SQ study. Once again, the 
items that demonstrated stability were those that 
intuitively make sense (ie. established good rapport). On 
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the other hand, items such as "call supervisee by name one 
time per session*' did not maintain significance after the 
revision. Results of this study should be interpreted 
within the limits of the project. One limitation is that 
the sample was drawn from a single university. When 
comparing the SQ and SQ-R, a limitation is that the prior 
study sampled only "beginning level*' counselors, whereas the 
latter study combined supervisees from several levels. 

Knowledge of supervisor behaviors that correlate with 
perceived effective supervision, and having an instrument 
that evidences stability despite revisions begins to place 
some structure into a young, complex process. As the field 
of supervision research continues to benefit from the 
testing of more comprehensive models, it is important to. not 
lose sight of the few models and instruments that may appear 
more simplistic but that have a high utility index. 
Continued evaluation and research on such practical 
instruments can prove highly useful as the complex 
supervision process becomes more refined. 
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